Age prediction from coronary angiography using a deep neural network: Age as a potential label to extract prognosis-related imaging features

Coronary angiography (CAG) is still considered the reference standard for coronary artery assessment, especially in the treatment of acute coronary syndrome (ACS). Although aging causes changes in coronary arteries, the age-related imaging features on CAG and their prognostic relevance have not been fully characterized. We hypothesized that a deep neural network (DNN) model could be trained to estimate vascular age only using CAG and that this age prediction from CAG could show significant associations with clinical outcomes of ACS. A DNN was trained to estimate vascular age using ten separate frames from each of 5,923 CAG videos from 572 patients. It was then tested on 1,437 CAG videos from 144 patients. Subsequently, 298 ACS patients who underwent percutaneous coronary intervention (PCI) were analysed to assess whether predicted age by DNN was associated with clinical outcomes. Age predicted as a continuous variable showed mean absolute error of 4 years with R squared of 0.72 (r = 0.856). Among the ACS patients stratified by predicted age from CAG images before PCI, major adverse cardiovascular events (MACE) were more frequently observed in the older vascular age group than in the younger vascular age group (p = 0.017). Furthermore, after controlling for actual age, gender, peak creatine kinase, and history of heart failure, the older vascular age group independently suffered from more MACE (hazard ratio 2.14, 95% CI 1.07 to 4.29, p = 0.032). The vascular age estimated based on CAG imaging by DNN showed high predictive value. The age predicted from CAG images by DNN could have significant associations with clinical outcomes in patients with ACS.


Introduction
Coronary artery angiography (CAG) is still considered the reference standard for definitive diagnosis of coronary artery disease [1], especially for the treatment of acute coronary syndrome (ACS) [2,3], even though non-invasive testing has become more widespread [4]. However, CAG is an invasive test with the risk of complications such as bleeding and stroke [5], and when it is performed, it is therefore desirable to obtain as much useful information as possible for patient evaluation. To address this issue, we focused on the estimation of vascular age using CAG imaging in this study. Vascular age is a concept in relation to the hypothesis that the conversion of chronological age to age derived from vascular imaging features will lead to more accurate assessment of an individual's cardiovascular risk. Although coronary artery calcium (CAC) scoring by computed tomography (CT) [6] and carotid intima-media thickness (CIMT) [7] assessment by B-mode ultrasonography can be used to define vascular age [8], it is not clearly known whether CAG contains useful age-related imaging features. Recently, deep neural networks (DNNs) have been utilized to analyse various types of images [1,9], including data interpretation that is difficult for humans, such as predicting age and gender from electrocardiograms [10]. The purpose of this study was to develop a deep learning neural network to estimate vascular age based on coronary angiographic imaging and to examine the clinical usefulness of this age prediction.

Study design and coronary artery angiography acquisition
The study design was a single-centre retrospective analysis. Consecutive patients aged �18 years who underwent standby diagnostic CAG for any reason between January 2010 and December 2015 at The University of Tokyo Hospital were reviewed. Patients undergoing PCI or CABG after index CAG were included in the study. Patients who underwent PCI with diagnostic CAG (i.e., ad-hoc PCI) were excluded. CAGs that a highly trained cardiologist judged as showing insufficient contrast effect were excluded after image processing, and, if a wire or balloon used for PCI or measurement of fractional flow reserve was captured on the images, these images were manually excluded, as were contrast images of coronary artery bypass. There was no other information in the image like the ECG signal or date of birth that could have also been used by the neural network to estimate age. Only those CAGs evaluated as "without coronary artery stenosis greater than 75%" were included. All CAG procedures in the included patients were performed as standard procedures in The University of Tokyo Hospital. This study was conducted in accordance with the revised Declaration of Helsinki and was approved by our institutional and local ethics committees (reference number 2650-(13)). Informed consent was obtained in the form of an opt-out selection on the web-site.  [20.1%]). No patients were included in more than one of the three datasets. The training dataset was used to train the neural network and the hyper parameters were then tuned using the validation dataset. The prediction accuracy of the final neural network was tested using the test dataset.

Coronary artery angiography and chronological age datasets
CAG videos of different time length were acquired as Digital Imaging and Communication in Medicine (Dicom) files recorded at 15 frames per second. The image size was 512 × 512 pixels and each pixel contained density information from 0 to 255. Therefore, the structure of a CAG video was the 3D matrix (T nk , Y ni , X nj ), where nk represents the number of frames in each CAG video, ni is the pixel position from 0 to 512, and nj is the pixel position from 0 to 512. Since some CAG videos contained frames with high information and others with low information, the frames with high information content were extracted using edge filters [11,12]. A total of ten frames were used from each video, with the frames with the largest edge being selected, and each final video was represented by a matrix of (10, 512, 512).
The patients' chronological ages, defined as the number of years since birth, were obtained from the catheterization report.

Development of the neural network for vascular age estimation
First, we defined vascular age as the age estimated based on a patient's CAG imaging by the neural network. A two-dimensional convolutional neural network (2D-CNN) to predict age from CAG was implemented using transfer learning and fine-tuning techniques in Python [13] with 4 sets of Nvidia Tesla A100 80 GB graphics processing unit (NVIDIA Corporation, Santa Clara, USA). Briefly, the methods utilized a pre-trained neural network to reduce the training time and the amount of data required for training. A neural network can learn much faster and with substantially fewer training examples if transfer learning and fine-tuning are employed, rather than training from scratch [14,15]. We adopted the EfficientNet [16], a commonly used 2D-CNN architecture, as the pre-trained neural network. The pre-trained weights on ImageNet for Effi-cientNet were downloaded from https://github.com/Cadene/pretrained-models.pytorch [17].
A single video represented by a (10, 512, 512) matrix was treated as ten separate frames, each of which was given a chronological age label and reshaped to (10, 600, 600) for input into the neural network. The training dataset was used to train the neural network to predict age as a continuous variable by minimizing the mean squared error (MSE) between predictions and ground truth age labels. A classification neural network to detect age �65 was also created using a similar neural network with binary cross-entropy loss [18]. These procedures used an Adam optimizer [19] with a batch size of 16 for 100 epochs. The learning rate was set to an initial value of 0.00001, 0.000001, 0.0000001, or 0.0000005 and then gradually reduced, with the initial learning rate with the lowest MSE or binary cross-entropy loss at the time of inference being used. The learning rate was reduced by a factor of two if the validation loss plateaued after three epochs. If the loss did not decrease for five consecutive epochs, the neural network training was stopped, even if 100 epochs had not been completed, and the neural network weights at the lowest validation loss were saved.

Performance evaluation
The trained neural network was applied to the CAG images in the test dataset and the predicted ages were calculated as continuous variables. These test dataset predictions were used to evaluate the predictive performance of the neural network on a per-CAG procedure basis. The per-frame assessment depended on the results of a single-frame, the per video assessment was the average of the ten per-frame assessments, and the per-CAG assessment was the average of the per-video assessments [20]. The correlation coefficient R, R squared, and mean average error were used to evaluate the neural network. The outputs of the neural network were also evaluated as multi-group to determine the accuracy of the predicted age within the age groups of 18 to 50, 50 to 70, and over 70 years. For classification neural network to detect age �65, the accuracy, sensitivity, and specificity with a cut-off value of 0.5, and the area under the receiver operating characteristics curve, were calculated. Additional subgroup analyses were performed for target coronary artery (right or left) and gender (male or female). The gradient-weighted class activation mapping (Grad-CAM) method was used to visualize the regions affecting the interpretations of the developed neural network [21].

Associations between estimated vascular age and clinical outcome in patients with ACS
Associations between predicted age and clinical outcomes in patients with ACS were examined. This analysis included 298 ACS patients who underwent PCI at our institution between 2010 and 2015 and whose video acquisitions were available. ACS was defined according to the universal definition [3]. The exclusion criteria were: (1) the second or more than second PCI performed during the study period, (2) patients with a history of coronary artery bypass grafting, (3) patients without follow-up information. Finally, 298 ACS patients were used to evaluate the associations between predicted age and clinical outcomes in patients with ACS. All individual CAG images were evaluated using a network pre-trained to estimate vascular age. The predicted age was obtained from each pre-PCI image as a continuous variable, and ACS patients were divided into two groups: a younger vascular age group (predicted age <65) and an older vascular age group (predicted age ≧65) [22]. The major adverse cardiovascular events (MACE) were compared between a younger vascular age group and an older vascular age group. MACE were defined as cardiac death, ACS, non-fatal cerebral infarction, and admission for heart failure. The index date was the date when the PCI was performed.
Hypertension was defined as a systolic blood pressure >140 mmHg, diastolic blood pressure >90 mmHg, or medical treatment for hypertension [23]. Diabetes mellitus was defined as haemoglobin A1c >6.5% or treatment for diabetes mellitus [24]. Dyslipidaemia was defined as total cholesterol >220 mg/dl, low-density lipoprotein cholesterol >140 mg/dl, or treatment for hyperlipidaemia. Shock was defined as systolic blood pressure <90 mmHg, use of vasopressors to maintain blood pressure, or attempted cardiopulmonary resuscitation [24]. Cerebral infarction was defined as an acute episode of focal or global neurological dysfunction caused by brain, spinal cord, or retinal vascular injury resulting from haemorrhage or infarction [25].

Statistical analysis
Data are expressed as mean ± standard deviation or number (percentage). Categorical variables were compared using the chi squared test (or Fisher's exact test for small samples). Normally distributed continuous variables were compared using Student's t test and abnormally distributed continuous variables were compared using the Mann-Whitney U test. Event free survival curves were constructed using the Kaplan-Meier method, and statistical differences between curves were assessed using the log-lank test. P values < 0.05 were considered statistically significant. A multivariate Cox regression analysis was performed to investigate associations between in-hospital complications and MACE after controlling for known clinical confounders. Hazard ratios (HRs) and 95% confidence intervals (CI) were calculated. All statistical analyses were performed using R (R Foundation for Statistical Computing, Vienna, Austria).

Patient selection
A total of 7,360 CAG videos from 937 CAG procedures performed on 716 patients between January 2010 and December 2015 were included. In total, 106 patients underwent multiple CAG procedures. The characteristics of the patients included in this study are shown in

Performance in the age prediction
As the output of the neural network was a continuous variable, the statistic of absolute error was calculated together with the overall correlation and the explained variance (R squared). For the test dataset, the mean absolute error was 4 years and R squared was 0.72 (r = 0.856). A scatter plot of chronological age versus predicted age is presented in Fig 2A. For the multigroup classification into age groups of 18 to 50, 50 to 70, and 70 years and above, the overall accuracy was 68% (Fig 2B). For detection of age �65, the AUC was 0.839 with a sensitivity of 74%, specificity of 76%, and accuracy of 75% (S1 Fig). Subgroup analysis according to target vessel showed R squared of 0.69 (r = 0.830) in the RCA group and 0.73 (r = 0.846) in the LCA group (Fig 3A and 3B). Gender analysis showed R squared of 0.68 (r = 0.826) in the male group and 0.83 (r = 0.910) in the female group (Fig 3C and 3D).

Visualization of neural network decision making
Grad-CAM analysis demonstrated that the neural network focused on the entire coronary artery limbus to predict age from CAG (Fig 4).

Associations between predicted age and clinical outcomes in patients with ACS
The clinical characteristics of the younger vascular age group and older vascular age group are shown in Table 2 for the 298 ACS patients used to determine associations between predicted age obtained from pre-PCI CAG images and clinical outcomes. The mean absolute error between predicted age and chronological age was 3 years with an R squared of 0.38 (r = 0.615). ST-elevated myocardial infarction, male sex, and peak-CK were higher in the younger vascular age group than in the older vascular age group, and the percentage of chronological age ≧65 years was 37.7% in the younger vascular age group and 75.5% in the older vascular age group ( Table 2). The clinical outcomes of the two groups are shown in Table 3 Table 4. The older vascular age group showed a significant association with MACE (hazard ratio 2.14, 95% CI 1.07  to 4.29, p = 0.032) after controlling for actual age, gender, peak creatine kinase, and history of heart failure (versus younger vascular age group).

Discussion
In this study, we developed and validated a deep learning algorithm based on a 2D-CNN for the age prediction using CAG images. We demonstrated that the predicted age had promising potential for predicting patient outcomes, and we also showed which coronary artery feature most influenced the predictions of the neural network in the Grad-CAM analysis. MACE were more frequently observed in the older vascular age group than in the younger vascular age group (p = 0.017). Furthermore, the older vascular age group was significantly associated with MACE (hazard ratio 2.14, 95% CI 1.07 to 4.29, p = 0.032) after controlling for known clinical risk factors. To the best of our knowledge, this is the first study to develop a neural network for predicting age based on CAG imaging and to use Grad-CAM to demonstrate the coronary artery feature that may be essential for the neural network decision-making. Neural networks have been applied to data from various modalities for the purposes of age prediction [10,26]. In the field of cardiology, neural networks for predicting chronological age from ECGs or chest x-rays have been developed and applied to clinical studies to investigate the clinical implications of predicted age. In such a study, a model for predicting chronological age from ECGs showed an R squared value of 0.70 and predicted age was found to be associated with patient comorbidity [10]. Another study created a model for predicting age from chest x-rays and showed R squared of 0.92 and found that predicted age was associated with outcomes of patients with heart failure [26]. However, to the best of our knowledge, so far, there was no study that attempted to predict age from CAG images, which involves a set of images containing time series information. Our neural network to predict age from CAG images showed an R squared of 0.72, and we believe that its accuracy is sufficiently high compared with previous studies. Furthermore, while most previous studies used training data with more than 100,000 samples to create their models, our neural network was trained only using several thousand videos. This suggests that the method of treating the video on a frame-byframe basis and then finally averaging the output of the neural network may have contributed to efficient extraction of information contained in the CAG video [20]. It also suggests that the age-related imaging features contained in the CAG imaging are robust.
Although several studies have proposed CAC scoring on coronary CT [27,28] (estimating the degree of coronary artery calcification) as an indicator of aging, and vascular tortuosity is known to show age-related change in CAG [29], it had not been clearly established which CAG imaging features allowed age prediction. In addition, it is particularly important to identify clinical findings that are potentially modifiable. In this study, we used the Grad-CAM method to provide the visualization of the CAG image regions that may be essential for age prediction by the neural network and found that the neural network may focus on the limbus of the coronary arteries, rather than on local coronary artery features. This result may suggest that the neural network is correctly extracting information from the coronary artery images by themselves (depending on the neural network training, the neural network could extract information from the ribs, lung fields, and cardiac shadow). It is also possible that the coronary artery limbus may contain age-related information, which was not previously reported.
In our analysis of ACS patients who underwent PCI at our institution, we showed that when the patients were stratified according to the vascular age estimated from pre-PCI CAG images, the stratification showed a significant association with long-term outcomes. This suggests that pre-treated coronary artery status may provide useful information reflecting the patient prognosis. In general, the training of a neural network requires a large quantity of labelled data and the cost of labelling sufficient numbers of data entries to create the network can be enormous. However, the method used in this study to extract imaging features relevant to age, which has a well-established association with prognosis [30,31], may have a potential for creating prognostic models.
The potential clinical implications of the present study should be noted. The age predicted using CAG imaging by neural network had high predictive value. Our study is particularly relevant because it adds further value to CAG and may lead to the exploration of new clinical findings that are potentially modifiable. For example, there may be a method for detecting The present study has the following limitations. First, because this was a single-centre retrospective study, there may have been a patient selection bias, which makes it difficult to generalize our results. Furthermore, as the model was validated with separate internal data, it is possible that the model performance might be lower with external data. Second, in the present study, there is a possibility that the neural network model might be trained on biased data because of the lack of information on comorbidities or the reason for CAG. Since we enrolled patients undergoing standby diagnostic CAG for any reason to train the model, there might be no controls, in other words, "healthy" subjects. We also validated the impact of predicted age on clinical outcomes of ACS patients, whereas the data used to train the neural network did not focus on ACS. Ideally, an age prediction by the neural network trained on coronary arteries of ACS patients should be developed, which seems to be difficult because of the limited number of ACS cases. In addition, the cut off age for younger and older vascular age groups was set at 65 years old, based on the average age of the ACS patients included in this study. It is difficult to generalize the method used in this study because, as mentioned above, it is the result of a single-centre retrospective study with a limited number of patients. In fact, the average age for PCI patients in Japanese clinical practice is known to be higher [32]. In the future, the usefulness of age predicted from CAG images should be explored using a large amount of CAG images and clinical data from ACS patients. Finally, although our Grad-CAM analysis showed that the neural network focused on the limbus of the coronary artery for both the LCA and RCA, we could not clarify exactly what feature of the coronary arteries is essential for predicting vascular age from CAG images. Such a problem is inherent to the nature of deep learning, which is often called the "black box of deep neural network" [33], and technologies that can explain deep neural network criteria in more detail are required.

Conclusions
We developed a neural network to predict age based on CAG imaging and found that it showed high predictive value. The age predicted from CAG images by deep neural network could have significant associations with clinical outcomes in patients with ACS.